Learning Deep State Representations With Convolutional Autoencoders

Advances in artificial intelligence algorithms and techniques are quickly allowing us to create artificial agents that interact with the real world. However, these agents need to maintain a carefully constructed abstract representation of the world around them [9]. Recent research in deep reinforcement learning attempts to overcome this challenge. Mnih et al. [24] at DeepMind and Levine et al. [18] demonstrate successful methods of learning deep end-to-end policies from high-dimensional input. In addition, Böhmer et al. [1] and Mattner et al. [22] extract deep state representations that can be used with traditional value function approximation algorithms to learn policies. We present a model that discovers low-dimensional deep state representations in a similar fashion to the deep fitted Q algorithm [1]. A plethora of function approximation techniques can be used in the lower dimension space to obtain the Qfunction. To test our algorithms, we run several experiments on 80× 20 images taken from a 10 × 2 grid world and show that convolutional autoencoders can be trained to obtain deep state representations that are almost as good as knowing the ground-truth state.